Correction to: 'Raising the value of research studies in psychological science by increasing the credibility of research reports: The Transparent Psi Project' (2023) by Kekecs et al.

[This corrects the article DOI: 10.1098/rsos.191375.][This corrects the article DOI: 10.1098/rspb.2022.2500.].

Analyses of response time data in the same-different task.

The Same-Different task presents two stimuli in close succession and participants must indicate whether they are completely identical or if there are any attributes that differ. While the task is simple, its results have proven difficult to explain. Notably, response times are characterized by a fast-same effect whereby Same responses are faster than Different responses even though identical stimuli should be exhaustively processed to be accurate. Herein, we examine a little more than a quarter million response times (N = 255,744) obtained from 327 participants who participated in one of 14 variants of the task involving minor changes in the stimuli or their durations. We performed distribution fitting and analyzed estimated parameters stemming from the ex-Gaussian, lognormal, and Weibull distributions to infer the cognitive processing characteristics underlying this task. The results exclude serial processing of the stimuli and do not support dual-route processing. The fast-same effect appears only through a shift of the entire response time distributions, a feature impossible to detect solely with mean response time analyses. An attention-modulated process driven by entropy may be the most adequate model of the fast-same effect. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project.

The low reproducibility rate in social sciences has produced hesitation among researchers in accepting published findings at their face value. Despite the advent of initiatives to increase transparency in research reporting, the field is still lacking tools to verify the credibility of research reports. In the present paper, we describe methodologies that let researchers craft highly credible research and allow their peers to verify this credibility. We demonstrate the application of these methods in a multi-laboratory replication of Bem's Experiment 1 (Bem 2011 J. Pers. Soc. Psychol. 100, 407-425. (doi:10.1037/a0021524)) on extrasensory perception (ESP), which was co-designed by a consensus panel including both proponents and opponents of Bem's original hypothesis. In the study we applied direct data deposition in combination with born-open data and real-time research reports to extend transparency to protocol delivery and data collection. We also used piloting, checklists, laboratory logs and video-documented trial sessions to ascertain as-intended protocol delivery, and external research auditors to monitor research integrity. We found 49.89% successful guesses, while Bem reported 53.07% success rate, with the chance level being 50%. Thus, Bem's findings were not replicated in our study. In the paper, we discuss the implementation, feasibility and perceived usefulness of the credibility-enhancing methodologies used throughout the project.

Is the fast-same phenomenon that fast? An investigation of identity priming in the same-different task.

The same-different task is a classic paradigm that requires participants to judge whether two successively presented stimuli are the same or different. While this task is simple, with results that have been replicated many times, response times (RTs) and accuracy for both same and different decisions remain difficult to model. The biggest obstacle in modeling the task lies within its effect referred to as the fast-same phenomenon whereby participants are much faster at responding "same" than "different," while most standard cognitive models predict the opposite. In this study, we investigated whether this effect is the result of identity priming activated by the first stimulus. We ran four variants of the same-different task in which identity priming is intended to be attenuated or cancelled in half of the trials. Results for all four variants show that a complete visual match between both stimuli is necessary to observe a fast-same effect and that hampering this relation attenuates same RTs while different RTs remained relatively unchanged. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Examining the Mechanisms of Internal and External Focus of Attention With Donders' Subtractive Method.

The goal of the current study was to measure the processing demands on the stages of information processing with internal and external foci of attention. Participants completed simple and two-choice reaction time tasks with internal and external foci of attention. Donders' subtraction method was used to isolate the cumulative duration of stages unique to simple and choice reaction time tasks. Mean reaction time was comparable with internal and external foci of attention in simple and two-choice reaction time tasks. These results suggest that processing demands were comparable with internal and external foci of attention. We hypothesize that there was not a processing advantage for an external focus in simple reaction time because the required movements had low movement complexity.

Analysis of proportions using arcsine transform with any experimental design.

Introduction: Exact tests on proportions exist for single-group and two-group designs, but no general test on proportions exists that is appropriate for any experimental design involving more than two groups, repeated measures, and/or factorial designs. Method: Herein, we extend the analysis of proportions using arcsine transform to any sort of design. The resulting framework, which we have called Analysis of Proportions Using Arcsine Transform (ANOPA), is completely analogous to the analysis of variance for means of continuous data, allowing the examination of interactions, main and simple effects, post-hoc tests, orthogonal contrasts, et cetera. Result: We illustrate the method with a few examples (single-factor design, two-factor design, within-subject design, and mixed design) and explore type I error rates with Monte Carlo simulations. We also examine power computation and confidence intervals for proportions. Discussion: ANOPA is a complete series of analyses for proportions, applicable to any design.

Two case studies of very long-term retention.

Here, we present two case studies of extremely long-term retention. In the first, Richard C. Atkinson (RCA) had learned word sequences during experiments for his dissertation. Sixty-seven years later, RCA relearned the same words either in the original order or in a scrambled order. RCA reported no conscious awareness that the words were those used in the dissertation, but his relearning was considerably better for the words in the original order. In the second case study, Denis Cousineau had searched displays of objects for the presence of a target. The targets and foils had been novel at the beginning of training, and his search rate improved markedly over about 70 sessions. After 22 years, retraining showed retention of much of this gain in rate of search, and the rate was markedly faster than search for new objects with the same structure as the trained set. We consider interpretations of these case studies for our understanding of long-term retention.

Is perfectionism a killer of creative thinking? A test of the model of excellencism and perfectionism.

The standards that a person pursue in life can be set in a rigid or flexible way. The recent literature has emphasized a distinction between high and realistic standards of excellence, from high and unrealistic standards of perfection. In two studies, we investigated the role of striving towards excellence (i.e., excellencism) and striving towards perfection (i.e., perfectionism) in relation to divergent thinking, associative thinking, and openness to experience, general self-efficacy, and creative self-beliefs. In Study 1, 279 university students completed three divergent thinking items, which called for creative uses of two common objects and to name original things which make noise. A measure of openness to experience was included. Results from multiple regression indicated that participants pursuing excellence tended to generate more answers and more original ones compared with those pursuing perfection. Openness to experience was positively associated to excellencism and negatively associated to perfectionism. In Study 2 (n = 401 university students), we replicated these findings and extended them to associative tasks requiring participants to generate chains of unrelated words. Additional individual differences measures included general self-efficacy, creative self-efficacy, and creative personal identity. The results suggested that excellencism was associated with better performance on divergent thinking and associative tasks, compared with perfectionism. Excellencism was positively associated with all four personality variables, whereas perfectionism was significantly and negatively associated with openness to experience only. Implications for the distinction between perfectionism and excellencism with respect to creative indicators are discussed. In addition, the paradoxical finding that perfection strivers had high creative self-efficacy and creative personal identity but lower openness to experience and poorer performance on objective indicators of creative abilities is discussed.

Different measures of holistic face processing tap into distinct but partially overlapping mechanisms.

Holistic processing, which includes the integration of facial features and analysis of their relations to one another, is a hallmark of what makes faces 'special'. Various experimental paradigms purport to measure holistic processing but these have often produced inconsistent results. This has led researchers to question the nature and structure of the mechanism(s) underlying holistic processing. Using an individual differences approach, researchers have examined relations between various measures of holistic processing in an attempt to resolve these questions. In keeping with this, we examined relationships between four commonly used measures of holistic face processing in a large group of participants (N = 223): (1) The Face Inversion Effect, (2) the Part Whole Effect (PWE), (3) the Composite Face Effect, and (4) the Configural Featural Detection Task (CFDT). Several novel methodological and analytical elements were introduced, including the use of factor analysis and the inclusion of control conditions to confirm the face specificity of all of the effects measured. The four indexes of holistic processing derived from each measure loaded onto two factors, one encompassing the PWE and the CFDT, and one encompassing the CE. The 16 conditions tested across the four tasks loaded onto four factors, each factor corresponding to a different measure. These results, together with those of other studies, suggest that holistic processing is a multifaceted construct and that different measures tap into distinct but partially overlapping elements of it.

The cognitive architecture of processes responsible to assess similarity and clarity in a comparison task.

When asked to compare two stimuli, participants are on average faster to respond Same than Different, an effect coined the fast-same. The dual-process theory argues that information about similarity is processed in priority over any other type of information, causing the fast-same effect. We tested this serial architecture of cognitive processes using a double factorial paradigm, suitable for a Systems Factorial Technology (SFT) analysis. Twenty participants completed a task in which they compared two letters, which were varied on two dimensions: the similarity and the clarity of the letters. Their task was to indicate if the second letter was the Same as the second letter (ranging from identical and clear to similar and slightly blurry) or if it was Different (if the stimuli were either dissimilar or very blurry). The SFT results show that most participants processed the information in serial, but in a mixed order. In other words, for some trials, participants processed similarity first, and for some other trials, they processed clarity first. This implies that participant indeed processed information in serial in the comparison task, but that it does not cause the fast-same effect.

Project DyAdd: Non-linguistic Theories of Dyslexia Predict Intelligence.

Two themes have puzzled the research on developmental and learning disorders for decades. First, some of the risk and protective factors behind developmental challenges are suggested to be shared and some are suggested to be specific for a given condition. Second, language-based learning difficulties like dyslexia are suggested to result from or correlate with non-linguistic aspects of information processing as well. In the current study, we investigated how adults with developmental dyslexia or ADHD as well as healthy controls cluster across various dimensions designed to tap the prominent non-linguistic theories of dyslexia. Participants were 18-55-year-old adults with dyslexia (n = 36), ADHD (n = 22), and controls (n = 35). Non-linguistic theories investigated with experimental designs included temporal processing impairment, abnormal cerebellar functioning, procedural learning difficulties, as well as visual processing and attention deficits. Latent profile analysis (LPA) was used to investigate the emerging groups and patterns of results across these experimental designs. LPA suggested three groups: (1) a large group with average performance in the experimental designs, (2) participants predominantly from the clinical groups but with enhanced conditioning learning, and (3) participants predominantly from the dyslexia group with temporal processing as well as visual processing and attention deficits. Despite the presence of these distinct patterns, participants did not cluster very well based on their original status, nor did the LPA groups differ in their dyslexia or ADHD-related neuropsychological profiles. Remarkably, the LPA groups did differ in their intelligence. These results highlight the continuous and overlapping nature of the observed difficulties and support the multiple deficit model of developmental disorders, which suggests shared risk factors for developmental challenges. It also appears that some of the risk factors suggested by the prominent non-linguistic theories of dyslexia relate to the general level of functioning in tests of intelligence.

The fast-same effect of an exclusive-OR task.

Participants are faster to decide that two stimuli are identical than to decide that they are different. Opposing theories suggested that this fast-same effect is either due (a) to a response bias toward similarity or (b) to facilitation caused by the repetition of the stimuli attributes. Although both theories predict the fast-same effect in a conventional same-different task, they make distinct predictions for tasks in which response bias is removed. In such tasks, the bias theory predicts that the fast-same would disappear whereas the facilitation theory predicts that the fast-same would remain. We tested those hypotheses using a same-different task in which participants had to indicate if all the attributes of the stimuli were matching or all were mismatching by pressing one response key, or if some attributes were matching and some were mismatching, by pressing another response key. We call this an exclusive-OR same-different task. Results show that participants were much faster in the "all-matching" condition compared with the "all-mismatching" condition, therefore supporting the facilitation theory. A fit of the linear ballistic accumulator model to the observed data provide additional supports that the fast-same effect is not caused by bias, but by a faster accumulation rate of evidence in the "all-matching" condition. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Cognitive architecture and capacity of the cognitive system responsible for Same - Different judgments.

Participants tend to match identical pairs of stimuli faster than different pairs. Despite many endeavours to explain this fast-same effect, there is still no theoretical consensus. A potential reason for the lack of consensus is that the cognitive architecture and capacity underlying such phenomenon is assumed and not formally tested. For example, the dual-process approach suggests that Same responses arise from a parallel treatment, whereas Different responses arise from a serial treatment. It also suggests that in both conditions, the capacity of the process is unaffected by workload (unlimited capacity). Alternative approaches argue that the fast-same effect can be explained by parallel or coactive architectures with channels working in either limited or super capacity. In this study, we formally assess the architecture (three possibilities: serial, parallel and coactive) and the capacity (three possibilities: unlimited, limited and super-capacity) of the cognitive system in a Same-Different task using Systems Factorial Technology (SFT). We recruited twenty participants to perform a double-factorial task lasting four sessions. Because of the lack of effectiveness of the blurring manipulation, we cannot draw a strong conclusion about the cognitive architecture. As for the capacity, the results show that it is mostly limited for the majority of participants. However, between 300 and 500 ms, participants tend to have a much stronger processing capacity in the Same condition compared to the Different condition. This short but strong burst of activity for identical stimuli might explain the fast-same effect.

Sequential sampling models of same-different data and how they explain the fast-same effect.

There have been many empirical endeavors to explain the fact that participants correctly identify same stimuli faster than different stimuli. Despite a large body of literature, a conclusive and unanimous consensus to explain this effect is lacking. The present study offers a new perspective on the same-different task using best-fitting parametric values of 2 sequential sampling models (linear ballistic accumulator [LBA] and drift-diffusion model [DDM]) to evaluate the contribution of 3 theoretical concepts (starting point, accumulation rate, and base time) to the fast-same effect. Eighteen participants were recruited to perform 3 tasks in which they compared letter strings (a) in a normal setting, the control task; (b) with lower contrast, the contrast task; and (c) based on nominal identity, the case task. Both the LBA and the DDM suggests that participants have faster base time in the same condition compared with the different condition. They also suggest that the number of letters and the number of differences influence the accumulation rate. These results suggest that the cognitive processes do not accumulate information as efficiently when the workload is increased. Finally, the limited predictive power of the best-fitting models suggests that better theoretical grounds are needed to identify fundamental cognitive concepts underlying same-different judgments. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Can categorical knowledge be used in visual search?

Smith, Redford, Gent, and Washburn (2005) have proposed a new categorization paradigm called the visual-search categorization task to study how display size affects categorization performance. Their results show that, in a wide range of conditions, category knowledge collapses as soon as multiple stimuli are simultaneously displayed in a scene. This result is surprising and important considering that humans parse and categorize objects from complex scenes on a daily basis. However, Smith et al. only studied one kind of category structure. This article presents the results of three experiments exploring the effect of display size on perceptual categorization as a function of category structure. We show that rule-based and information-integration categories are differently affected by display size in the visual search categorization task. For rule-based structures, target-present and target-absent trials are not much affected by display size. However, the effect of display size is bigger for information-integration category structures, and much more pronounced for target-absent trials than for target-present trials. A follow-up experiment shows that target redundancy (i.e., having more than one target in the display) does not improve performance with information-integration category structures. These results suggest that categories may be learned differently depending on their underlying structure, and that the resulting category representation may influence performance in the visual search categorization task.

Manipulating the Alpha Level Cannot Cure Significance Testing.

We argue that making accept/reject decisions on scientific hypotheses, including a recent call for changing the canonical alpha level from p = 0.05 to p = 0.005, is deleterious for the finding of new discoveries and the progress of science. Given that blanket and variable alpha levels both are problematic, it is sensible to dispense with significance testing altogether. There are alternatives that address study design and sample size much more directly than significance testing does; but none of the statistical tools should be taken as the new magic method giving clear-cut mechanical answers. Inference should not be based on single studies at all, but on cumulative evidence from multiple independent studies. When evaluating the strength of the evidence, we should consider, for example, auxiliary assumptions, the strength of the experimental design, and implications for applications. To boil all this down to a binary decision based on a p-value threshold of 0.05, 0.01, 0.005, or anything else, is not acceptable.

Varieties of Confidence Intervals.

Error bars are useful to understand data and their interrelations. Here, it is shown that confidence intervals of the mean (CI M s) can be adjusted based on whether the objective is to highlight differences between measures or not and based on the experimental design (within- or between-group designs). Confidence intervals (CIs) can also be adjusted to take into account the sampling mechanisms and the population size (if not infinite). Names are proposed to distinguish the various types of CIs and the assumptions underlying them, and how to assess their validity is explained. The various CIs presented here are easily obtained from a succession of multiplicative adjustments to the basic (unadjusted) CI width. All summary results should present a measure of precision, such as CIs, as this information is complementary to effect sizes.

Canadian Normative Data for Minimal Assessment of Cognitive Function in Multiple Sclerosis.

OBJECTIVE: The Minimal Assessment of Cognitive Function in Multiple Sclerosis (MACFIMS) is a consensus-based collection of neuropsychological tests that evaluate cognitive functioning in individuals with multiple sclerosis (MS). The tests are typically scored using each respective published test manual, leaving the examiner to make interpretations from norms derived from different American populations. Given demographic differences, this may lead to misinterpretation of findings in Canadians. Our goal was to establish both discrete and regression-based normative data for the MACFIMS based on a largely co-normed Canadian population to allow for improved psychometric interpretation. METHODS: MACFIMS data sets were aggregated from across three different Canadian cities (Ottawa, Toronto, and London), yielding a total of 330 healthy control participants from four different studies evaluating cognition in individuals with MS. Given the variety of contributing studies, there was variability in terms of the number of participants completing each measure. RESULTS: Both age-based discrete normative data and demographically adjusted (sex, age, and education) regression-based formulae were established. The demographic variables varied in their contribution to each MACFIMS test in the regression models, predicting 0 to 18% of the variance. CONCLUSIONS: Provision of these regression-based formulae will allow for more accurate interpretation of Canadian-derived MACFIMS scores by allowing clinicians to correct for all relevant demographic variables simultaneously, leading to improved clinical decision making for individuals with multiple sclerosis.

An Unbiased Estimate of Global Interrater Agreement.

Assessing global interrater agreement is difficult as most published indices are affected by the presence of mixtures of agreements and disagreements. A previously proposed method was shown to be specifically sensitive to global agreement, excluding mixtures, but also negatively biased. Here, we propose two alternatives in an attempt to find what makes such methods so specific. The first method, RB , is found to be unbiased while at the same time rejecting mixtures, is detecting agreement with good power and is little affected by unequal category prevalence as soon as there are more than two categories.